Skip to content

perf: pre-fetch CSO external ID dictionary for O(1) import matching#442

Merged
JayVDZ merged 4 commits intomainfrom
feature/findmatchingcso-batch-prefetch
Apr 1, 2026
Merged

perf: pre-fetch CSO external ID dictionary for O(1) import matching#442
JayVDZ merged 4 commits intomainfrom
feature/findmatchingcso-batch-prefetch

Conversation

@JayVDZ
Copy link
Copy Markdown
Contributor

@JayVDZ JayVDZ commented Apr 1, 2026

Summary

  • ⚡ Replace N per-object database queries during import with a single bulk query that loads all CSO external ID mappings into an in-memory dictionary at import start
  • ⚡ FindMatchingCso now uses O(1) dictionary lookup (Lookup phase) followed by PK-based hydration (Hydrate phase), eliminating the LOWER() string-scan queries that dominated import time
  • 🏗️ Lookup/Hydrate separation provides a clean seam for future persistent caching (Worker performance: MVO update conversion + advanced caching strategies #436)

Integration test results (S8 MediumLarge, OpenLDAP)

Metric Before After Improvement
Avg per call 17.2ms 2.2ms ~8x faster
Steady-state ~17ms ~1.0ms ~17x faster
Total FindMatchingCso 5m 4s 22.8s ~13x faster
Pre-fetch load N/A 93–174ms Single query for 5,273 CSOs

All 6 S8 tests passed. All 2,463 unit tests pass.

Test plan

  • 10 new tests in ImportBatchPrefetchTests.cs (empty CS, re-import, mixed new/existing, mid-batch creation, repository methods)
  • All 2,463 existing tests pass across all 7 test projects
  • Integration test: S8 MediumLarge OpenLDAP — all 6 steps passed
  • Integration test: S8 MediumLarge Samba AD

Closes #440

🤖 Generated with Claude Code

JayVDZ and others added 4 commits April 1, 2026 16:18
…440)

Replace N per-object database queries during import with a single bulk
query that loads all CSO external ID mappings into an in-memory dictionary
at import start. FindMatchingCso now uses O(1) dictionary lookup (Lookup
phase) followed by PK-based hydration (Hydrate phase), eliminating the
LOWER() string-scan queries that dominated import time.

The Lookup/Hydrate separation provides a clean seam for future persistent
caching (#436) — the dictionary becomes the cache warm-up path, and the
hydration method becomes the cache-miss handler.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…e skill

Adds a new "Documentation Review and Update" section that runs before
changelog validation, ensuring all docs, C4/Mermaid diagrams, and
marketing site content are current before each release.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add three new directive blocks: plan-before-building (with tunnel-vision
breaker), demand-quality standards, and subagent strategy. Replace the
manual release checklist with a pointer to the /release skill. Minor
wording cleanups (branch naming, security section).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move detailed reference material out of the root CLAUDE.md (657 -> 362
lines, 45% reduction) into contextual sub-folder files that load only
when working in those directories:

- src/CLAUDE.md: code conventions, architecture rules, project locations,
  common development tasks
- .devcontainer/CLAUDE.md: commands, shell aliases, Docker workflows,
  dependency policy, environment setup, troubleshooting

Root retains all behavioural guardrails, critical requirements, and short
summaries with pointers to the sub-files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@JayVDZ JayVDZ merged commit f799dbd into main Apr 1, 2026
9 checks passed
@JayVDZ JayVDZ deleted the feature/findmatchingcso-batch-prefetch branch April 1, 2026 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: batch pre-fetch CSO external ID mappings during full import

1 participant